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DETAILED ACTION 

Continued Examination Under 37 CFR 1.114 

1. A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
02/13/2009 has been entered. 

Response to Arguments 

2. Applicant's arguments filled 02/13/2009 have been considered but are moot in 
view of the new ground(s) of rejection. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 1-33 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Stephen Bugaj et al in the publication titled Synchronized Multimedia Integration 
Language, November 1997, edited by Phillip Hoscha (09/09/1997). (Bugaj), in view of 
US Patent Publication 2002/01 12249 to Hendricks et al. (Hendricks). 

Regarding claim 1, Bugaj in section 2. lines 1-4 discloses SMIL (synchronized 
media integration language) for integrating a set of independent multimedia objects into 
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a synchronized multimedia presentation such as slide show synchronized with audio 
comments or a video synchronized with text stream wherein SMIL is an XML-based 
language (section 3 line 1). In section 4 Bugaj teaches the general syntax of a SMIL 
document comprising a header and a body wherein both parts contain elements and 
attributes which reads on "An XML-based stored in a computer-readable medium for 
encoding a visual cue for visual component of a multimedia presentation, wherein the 
XML-based element is structured for use by a computer to display the multimedia 
presentation including the visual component and the visual cue on a display surface of 
the computer, wherein the XML-based -element comprises". 

Bugaj teaches a layout section of a SMIL document including alternative layout 
elements embedded in a switch element used to determine the placement of the 
presentation (section 5, General Semantics, lines 1-6), however, Bugaj is silent on a 
visual element attribute that defines a visual representation of the visual cue; a spatial 
element attribute that defines spatial characteristics of the visual cue; and a_temporal 
art-element attribute that defines temporal characteristics of the visual cue, wherein the 
and spatial characteristics of the visual cue are defined relative to temporal and spatial 
characteristics of the associated visual component, and wherein the computer 
superimposes a display of the visual associated cue on the display of the computer over 
the visual component in the multimedia presentation using a visual appearance which is 
based on the visual representation of the visual cue as defined in the visual element 
attribute that defines visual representation of the visual cue, during a period of time 
which is based on the temporal characteristics of the visual cue as defined in the 
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temporal element attribute that defines temporal characteristics of the visual cue, and at 
a location over the associated visual element which is based on the spatial 
characteristics of the visual cue as defined in the spatial element attribute that defines 
spatial characteristics of the visual cue . 

The examiner maintains that it was well known in the art to provide " a visual 
element attribute that defines a visual representation of the visual cue; a spatial element 
attribute that defines spatial characteristics of the visual cue; and ajemporal art-element 
attribute that defines temporal characteristics of the visual cue, wherein the and spatial 
characteristics of the visual cue are defined relative to temporal and spatial 
characteristics of the associated visual component, and wherein the computer 
superimposes a display of the visual associated cue on the display of the computer over 
the visual component in the multimedia presentation using a visual appearance which is 
based on the visual representation of the visual cue as defined in the visual element 
attribute that defines visual representation of the visual cue, during a period of time 
which is based on the temporal characteristics of the visual cue as defined in the 
temporal element attribute that defines temporal characteristics of the visual cue, and at 
a location over the associated visual element which is based on the spatial 
characteristics of the visual cue as defined in the spatial element attribute that defines 
spatial characteristics of the visual cue ", as taught by Hendricks. 

In a similar field of endeavor Hendricks teaches a visual element attribute that 
defines a visual representation of the visual cue; a spatial element attribute that defines 
spatial characteristics of the visual cue; and atemporal art-element attribute that defines 
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temporal characteristics of the visual cue, wherein the and spatial characteristics of the 
visual cue are defined relative to temporal and spatial characteristics of the associated 
visual component (figure 2, labels 36, & 40, & Para. 0061, & 0113). 

In addition, Hendricks teaches wherein the computer superimposes a display of 
the visual associated cue on the display of the computer over the visual component in 
the multimedia presentation using a visual appearance which is based on the visual 
representation of the visual cue as defined in the visual element attribute that defines 
visual representation of the visual cue, during a period of time which is based on the 
temporal characteristics of the visual cue as defined in the temporal element ..attribute 
that defines temporal characteristics of the visual cue, and at a location over the 
associated visual element which is based on the spatial characteristics of the visual cue 
as defined in the spatial element attribute that defines spatial characteristics of the 
visual cue (figure 3, Labels 36, & 44, & Para. 0061 , & 0070, & 01 1 3). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Bugaj by specifically providing " a visual element 
attribute that defines a visual representation of the visual cue; a spatial element attribute 
that defines spatial characteristics of the visual cue; and ajemporal art-element attribute 
that defines temporal characteristics of the visual cue, wherein the and spatial 
characteristics of the visual cue are defined relative to temporal and spatial 
characteristics of the associated visual component, and wherein the computer 
superimposes a display of the visual associated cue on the display of the computer over 
the visual component in the multimedia presentation using a visual appearance which is 
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based on the visual representation of the visual cue as defined in the visual element 
attribute that defines visual representation of the visual cue, during a period of time 
which is based on the temporal characteristics of the visual cue as defined in the 
temporal element ..attribute that defines temporal characteristics of the visual cue, and 
at a location over the associated visual element which is based on the spatial 
characteristics of the visual cue as defined in the spatial element attribute that defines 
spatial characteristics of the visual cue ", as taught by Hendricks, for the purpose of 
providing interactive presentations thereby providing additional information on 
presentation to user while user is watching the presentation. 

Regarding Claim 2, Bugaj teaches schedule elements including begin and end 
times wherein duration is defined as the difference between the end times and begin 
time of an element (section 6.1 , General Semantics, line 3, & section 6.2 page 8, line 3) 
which meets "An XML-based element wherein the temporal characteristics include at 
least two of begin time, end time, and duration". 

Regarding Claim 3, Bugaj teaches an image element tagged as an XML media 
object element (section 6.4, Syntax) which inherently meets "An XML-based element, 
wherein the visual representation includes color". 

Regarding Claim 4, Bugaj teaches an image element (section 6.4, Syntax) which 
meets "An XML-based element, wherein .the visual representation includes shape". 

Regarding Claim 5, Bugaj teaches a layout section of a SMIL document including 
alternative layout elements embedded in a switch element used to determine the 
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placement of the presentation (section 5, General Semantics, lines 1-6) which meets 
"An XML-based element, wherein the spatial characteristics include position". 

Regarding Claim 6, Bugaj teaches a smile document for the newscast 
presentation illustrated in figure 7.1 page 26 wherein the layout and temporal elements 
are controlled by their associated attributes in hierarchal order (section 7.4 page 27) 
which meets "An XML-based element, wherein the XML-based element for the visual 
cue is nested within an XML- based element that defines the associated visual 
component". 

Regarding claim 7, Bugaj teaches a news broadcast on the web as shown in 
figure 7.1 to the left and right (section 7.4, page 26, lines 2-9) which meets "In an XML- 
based browser that displays synchronized multimedia presentations on a display of a 
computer to user a method for processing an XML-based element for visual cue 
associated with a visual component of the multimedia presentation comprising" 

In addition Bugaj teaches an XML data structure holding the elements of the of 
the newscast scenario written in XML code comprising elements stored in tags wherein 
each element has its own properly defined attribute including spatial and temporal 
characteristics to perform a presentation (page 27) which meets "storing information 
from the XML-based element concerning the visual component to which the visual cue 
is associated, together with the information from the XML-based element concerning 
visual representation". 

Bugaj teaches a layout section of a SMIL document including alternative layout 
elements embedded in a switch element used to determine the placement of the 
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presentation (section 5, General Semantics, lines 1-6), however, Bugaj is silent on 
spatial and temporal characteristics of the visual cue; and in synchronization with 
display of the visual component , displaying the visual cue with the visual representation 
specified, and in the spatial and temporal relationships specified by the spatial and 
temporal characteristics, wherein the defined temporal and spatial characteristics of the 
visual cue are relative to temporal and spatial characteristics of the associated visual 
component, and wherein the display of the visual is superimposed over the associated 
visual component in the multimedia presentation with a visual appearance based on the 
defined visual representation of the visual cue, during a period of time based on the 
defined temporal characteristics of the visual cue, and at a location over the associated 
visual element based on the defined spatial characteristics of the visual cue. 

The examiner maintains that it was well known in the art to provide "spatial and 
temporal characteristics of the visual cue; and in synchronization with display of the 
visual component , displaying the visual cue with the visual representation specified, and 
in the spatial and temporal relationships specified by the spatial and temporal 
characteristics, wherein the defined temporal and spatial characteristics of the visual 
cue are relative to temporal and spatial characteristics of the associated visual 
component, and wherein the display of the visual is superimposed over the associated 
visual component in the multimedia presentation with a visual appearance based on the 
defined visual representation of the visual cue, during a period of time based on the 
defined temporal characteristics of the visual cue, and at a location over the associated 
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visual element based on the defined spatial characteristics of the visual cue", as taught 
by Hendricks. 

In a similar field of endeavor Hendricks teaches spatial and temporal 
characteristics of the visual cue; and in synchronization with display of the visual 
component , displaying the visual cue with the visual representation specified, and in the 
spatial and temporal relationships specified by the spatial and temporal characteristics, 
wherein the defined temporal and spatial characteristics of the visual cue are relative to 
temporal and spatial characteristics of the associated visual component (figure 2, labels 
36, & 40, & Para. 0061, & 01 13). 

In addition, Hendricks teaches wherein the display of the visual is superimposed 
over the associated visual component in the multimedia presentation with a visual 
appearance based on the defined visual representation of the visual cue, during a 
period of time based on the defined temporal characteristics of the visual cue, and at a 
location over the associated visual element based on the defined spatial characteristics 
of the visual cue (figure 3, Labels 36, & 44, & Para. 0061 , & 0070, & 01 1 3). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Bugaj by specifically providing "spatial and 
temporal characteristics of the visual cue; and in synchronization with display of the 
visual component , displaying the visual cue with the visual representation specified, and 
in the spatial and temporal relationships specified by the spatial and temporal 
characteristics, wherein the defined temporal and spatial characteristics of the visual 
cue are relative to temporal and spatial characteristics of the associated visual 
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component, and wherein the display of the visual is superimposed over the associated 
visual component in the multimedia presentation with a visual appearance based on the 
defined visual representation of the visual cue, during a period of time based on the 
defined temporal characteristics of the visual cue, and at a location over the associated 
visual element based on the defined spatial characteristics of the visual cue", as taught 
by Hendricks, for the purpose of providing interactive presentations thereby providing 
additional information on presentation to user while user is watching the presentation. 

Regarding Claim 8, Bugaj teaches schedule elements with begin and end time 
wherein duration is the difference between the end time and begin time of an element 
(section 6.1, General Semantics, line 3 section 6.2 page 8, line 3) which meets "An 
XML-based browser, wherein the temporal characteristics include at least two of begin 
time, end time, and duration". 

Regarding Claim 9, Bugaj teaches an image element tagged as an XML media 
object element (section 6.4, Syntax) which inherently meets "An XML-based browser, 
wherein the visual representation includes color". 

Regarding Claim 10, Bugaj teaches an image attribute (section 6.4, Syntax) 
which meets "An XML-based browser, wherein the visual representation includes 
shape". 

Regarding Claim 11, Bugaj disclose a layout section of a SMIL document 
including alternative layout elements embedded in a switch element used to determine 
the placement of the presentation (section 5, General Semantics, lines 1-6) which 
meets "An XML-based browser, wherein the spatial characteristics include position". 
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Regarding Claim 12, Bugaj teaches an SMIL document for the newscast 
presentation illustrated in figure 7.1 page 26 wherein the layout and temporal elements 
are controlled by their associated attributes in hierarchal order (section 7.4 page 27) 
which meets "An XML-based browser, wherein the XML-based element for the visual 
cue is nested within an XML-based element that defines the associated visual 
component". 

Regarding claim 13, Bugaj teaches a news broadcast on the web as shown in 
figure 7.1 to the left and right (section 7.4, page 26, lines 2-9) which meets "A computer- 
readable storage medium storing computer executable process steps to .display a 
synchronized multimedia presentation on a display of a computer to a user, and to 
process an XML-based element for a visual cue associated with a visual component of 
the multimedia presentation wherein the computer-executable process step cause the 
computer to execute process steps comprising". 

In addition Bugaj teaches an XML data structure holding the elements of the of 
the newscast scenario written in XML code comprising elements stored in tags wherein 
each element has its own properly defined attribute including spatial and temporal 
characteristics to perform a presentation (page 27) which meets "a storing step to store 
information from the XML-based element concerning the element visual component to 
which the visual cue is associated, together with information from the XML-based 
element concerning visual representation". 

Bugaj teaches a layout section of a SMIL document including alternative layout 
elements embedded in a switch element used to determine the placement of the 
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presentation (section 5, General Semantics, lines 1-6), however, Bugaj is silent on 
spatial and temporal characteristics of the visual cue; and in synchronization with 
display of the visual component, a displaying step to display the visual cue with the 
visual representation in the spatial and temporal relationships specified by the spatial 
and temporal characteristics, wherein the defined temporal and spatial characteristics of 
the visual cue are relative to temporal and spatial characteristics of the associated 
visual component, and wherein the display of the visual cue is superimposed over the 
associated visual component in the multimedia presentation a visual appearance based 
on the defined visual representation of the visual cue, during a period of time based on 
the defined temporal characteristics of the visual cue, and at a location over the 
associated visual element based on the defined spatial characteristics of the visual cue. 

The examiner maintains that it was well known in the art to provide "spatial and 
temporal characteristics of the visual cue; and in synchronization with display of the 
visual component, a displaying step to display the visual cue with the visual 
representation in the spatial and temporal relationships specified by the spatial and 
temporal characteristics, wherein the defined temporal and spatial characteristics of the 
visual cue are relative to temporal and spatial characteristics of the associated visual 
component, and wherein the display of the visual cue is superimposed over the 
associated visual component in the multimedia presentation a visual appearance based 
on the defined visual representation of the visual cue, during a period of time based on 
the defined temporal characteristics of the visual cue, and at a location over the 
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associated visual element based on the defined spatial characteristics of the visual 
cue", as taught by Hendricks. 

In a similar field of endeavor Hendricks teaches spatial and temporal 
characteristics of the visual cue; and in synchronization with display of the visual 
component, a displaying step to display the visual cue with the visual representation in 
the spatial and temporal relationships specified by the spatial and temporal 
characteristics, wherein the defined temporal and spatial characteristics of the visual 
cue are relative to temporal and spatial characteristics of the associated visual 
component (figure 2, labels 36, & 40, & Para. 0061 , & 011 3). 

In addition, Hendricks teaches wherein the display of the visual cue is 
superimposed over the associated visual component in the multimedia presentation a 
visual appearance based on the defined visual representation of the visual cue, during a 
period of time based on the defined temporal characteristics of the visual cue, and at a 
location over the associated visual element based on the defined spatial characteristics 
of the visual cue (figure 3, Labels 36, & 44, & Para. 0061 , & 0070, & 01 1 3). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Bugaj by specifically providing "spatial and 
temporal characteristics of the visual cue; and in synchronization with display of the 
visual component, a displaying step to display the visual cue with the visual 
representation in the spatial and temporal relationships specified by the spatial and 
temporal characteristics, wherein the defined temporal and spatial characteristics of the 
visual cue are relative to temporal and spatial characteristics of the associated visual 
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component, and wherein the display of the visual cue is superimposed over the 
associated visual component in the multimedia presentation a visual appearance based 
on the defined visual representation of the visual cue, during a period of time based on 
the defined temporal characteristics of the visual cue, and at a location over the 
associated visual element based on the defined spatial characteristics of the visual 
cue", as taught by Hendricks, for the purpose of providing interactive presentations 
thereby providing additional information on presentation to user while user is watching 
the presentation. 

Regarding Claim 14, Bugaj teaches schedule elements with begin and end time 
wherein the duration is the difference between the end time and begin time of an 
element (section 6.1, General Semantics, line 3, & section 6.2 page 8, line 3) which 
meets "A computer-readable medium wherein the temporal characteristics include at 
least two of begin time, end time, and duration". 

Regarding Claim 15, Bugaj teaches an image element tagged as an XML media 
object element (section 6.4, Syntax) which inherently meets "A computer-readable 
medium, wherein the visual representation includes color". 

Regarding Claim 16, Bugaj teaches an image attribute (section 6.4, Syntax) 
which meets "A computer-readable medium, wherein the visual representation includes 
shape". 

Regarding Claim 17, Bugaj teaches a layout section of a SMIL document 
including alternative layout elements embedded in a switch element used to determine 
the placement of the presentation (section 5, General Semantics, lines 1-6) which 
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meets "A computer-readable medium, wherein the spatial characteristics include 
position". 

Regarding Claim 18, Bugaj teaches a SMIL document for the newscast 
presentation illustrated in figure 7.1 page 26 wherein the layout and temporal elements 
are controlled by their associated attributes in hierarchal order (section 7.4 page 27) 
which meets "A computer-readable medium, wherein the XML-based element for the 
visual cue is nested within an XML- based element that defines the associated visual 
component". 

Regarding claim 22, Bugaj teaches a news broadcast on the web as shown in 
figure 7.1 to the left and right (section 7.4, page 26, lines 2-9) which meets " A method 
for displaying a synchronized multimedia presentation on a display screen of a 
computer executing an XML-based browser, comprising". 

In addition Bugaj teaches an XML data structure holding the elements of the of 
the newscast scenario written in XML code comprising elements stored in tags wherein 
each element has its own properly defined attribute including spatial and temporal 
characteristics to perform a presentation (page 27) which meets "receiving XML-based 
data including an XM.L-based element for a visual cue together with an XM'L-based 
element for a visual component contained in the multimedia presentation, wherein the 
XML-based visual cue element is nested within the XML-based element for the 
associated visual component". 

Bugaj teaches a layout section of a SMIL document including alternative layout 
elements embedded in a switch element used to determine the placement of the 
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presentation (section 5, General Semantics, lines 1-6), however, Bugaj is silent on 
wherein the XMI based visual cue element includes attributes that define temporal and 
spatial relativity between a display of the visual cote and a display of the multimedia 
component; and displaying the synchronized multimedia presentation including the 
visual cue superimposed over the multimedia component in a temporal and spatial 
relationship defined by the attributes of the XML-based visual cue element. 

The examiner maintains that it was well known in the art to provide "wherein the 
XMI based visual cue element includes attributes that define temporal and spatial 
relativity between a display of the visual cote and a display of the multimedia 
component; and displaying the synchronized multimedia presentation including the 
visual cue superimposed over the multimedia component in a temporal and spatial 
relationship defined by the attributes of the XML-based visual cue element", as taught 
by Hendricks. 

In a similar field of endeavor Hendricks teaches wherein the XML-based visual 
cue element includes attributes that define temporal and spatial relativity between a 
display of the visual cote and a display of the multimedia component; and displaying the 
synchronized multimedia presentation including the visual cue superimposed over the 
multimedia component in a temporal and spatial relationship defined by the attributes of 
the XML-based visual cue element (figure 2, labels 36, & 40, & Para. 0061 , & 01 1 3). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Bugaj by specifically providing "wherein the 
XML-based visual cue element includes attributes that define temporal and spatial 
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relativity between a display of the visual cote and a display of the multimedia 
component; and displaying the synchronized multimedia presentation including the 
visual cue superimposed over the multimedia component in a temporal and spatial 
relationship defined by the attributes of the XML-based visual cue element", as taught 
by Hendricks, for the purpose of providing interactive presentations thereby providing 
additional information on presentation to user while user is watching the presentation. 

Regarding claim 23, Bugaj is silent on a visual element attribute that defines a 
visual representation of the visual cue; a spatial element attribute that defines spatial 
characteristics of the visual cue; and a temporal element attribute that defines temporal 
characteristics of the visual cue. 

The examiner maintains that it was well known in the art to provide "a visual 
element attribute that defines a visual representation of the visual cue; a spatial element 
attribute that defines spatial characteristics of the visual cue; and a temporal element 
attribute that defines temporal characteristics of the visual cue", as taught by Hendricks. 

In a similar field of endeavor Hendricks teaches a visual element attribute that 
defines a visual representation of the visual cue; a spatial element attribute that defines 
spatial characteristics of the visual cue; and a temporal element attribute that defines 
temporal characteristics of the visual cue (figure 3, Labels 36, & 44, & Para. 0061, & 
0070, & 01 13). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Bugaj by specifically providing "a visual element 
attribute that defines a visual representation of the visual cue; a spatial element attribute 
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that defines spatial characteristics of the visual cue; and a temporal element attribute 
that defines temporal characteristics of the visual cue", as taught by Hendricks, for the 
purpose of providing interactive presentations thereby providing additional information 
on presentation to user while user is watching the presentation. 

Regarding Claim 24, Bugaj teaches schedule elements with begin and end time 
wherein the duration is the difference between the end time and begin time of an 
element (section 6.1, General Semantics, line 3, & section 6.2 page 8, line 3) which 
meets "a method, wherein the temporal characteristics include at least two of begin 
time, end time, and duration". 

Regarding Claim 25, Bugaj teaches an image element tagged as an XML media 
object element (section 6.4, Syntax) which inherently meets "a method, wherein the 
visual representation includes at least one of a shape and a color of the visual cue". 

Regarding claim 26, Bugaj teaches a news broadcast on the web as shown in 
figure 7.1 to the left and right (section 7.4, page 26, lines 2-9) which meets "A computer- 
readable memory medium storing computer- executable process steps that cause a 
computer to display a synchronized multimedia presentation on a display screen of the 
computer which is executing an XML-based browser, wherein the computer-executable 
process steps comprise". 

In addition Bugaj teaches an XML data structure holding the elements of the of 
the newscast scenario written in XML code comprising elements stored in tags wherein 
each element has its own properly defined attribute including spatial and temporal 
characteristics to perform a presentation (page 27) which meets "receiving XML-based 
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data including an XML-based element for a visual cue together with an XML-based 
element for a visual component contained in the multimedia presentation, wherein the 
XML-based visual cue element is nested within the XML-based element for the 
associated visual component". 

Bugaj teaches a layout section of a SMIL document including alternative layout 
elements embedded in a switch element used to determine the placement of the 
presentation (section 5, General Semantics, lines 1-6), however, Bugaj is silent on 
wherein the XML-based visual cue element includes attributes that define temporal and 
spatial relativity between a display of the visual cue and a display of the visual 
component; and displaying the synchronized multimedia presentation including the 
visual cue superimposed over the visual component in a temporal and spatial 
relationship defined by the attributes of the XML-based visual cue element. 

The examiner maintains that it was well known in the art to provide "wherein the 
XML-based visual cue element includes attributes that define temporal and spatial 
relativity between a display of the visual cue and a display of the visual component; and 
displaying the synchronized multimedia presentation including the visual cue 
superimposed over the visual component in a temporal and spatial relationship defined 
by the attributes of the XML-based visual cue element", as taught by Hendricks. 

In a similar field of endeavor Hendricks teaches wherein the XML-based visual 
cue element includes attributes that define temporal and spatial relativity between a 
display of the visual cue and a display of the visual component; and displaying the 
synchronized multimedia presentation including the visual cue superimposed over the 
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visual component in a temporal and spatial relationship defined by the attributes of the 
XML-based visual cue element (figure 2, labels 36, & 40, & Para. 0061 , & 01 1 3). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Bugaj by specifically providing "wherein the 
XML-based visual cue element includes attributes that define temporal and spatial 
relativity between a display of the visual cue and a display of the visual component; and 
displaying the synchronized multimedia presentation including the visual cue 
superimposed over the visual component in a temporal and spatial relationship defined 
by the attributes of the XML-based visual cue element", as taught by Hendricks, for the 
purpose of providing interactive presentations thereby providing additional information 
on presentation to user while user is watching the presentation. 

Regarding claim 27, Bugaj is silent on a visual element attribute that defines a 
visual representation of the visual cue; a spatial element attribute that defines spatial 
characteristics of the visual cue; and a temporal element attribute that defines temporal 
characteristics of the visual cue. 

The examiner maintains that it was well known in the art to provide "a visual 
element attribute that defines a visual representation of the visual cue; a spatial element 
attribute that defines spatial characteristics of the visual cue; and a temporal element 
attribute that defines temporal characteristics of the visual cue", as taught by Hendricks. 

In a similar field of endeavor Hendricks teaches a visual element attribute that 
defines a visual representation of the visual cue; a spatial element attribute that defines 
spatial characteristics of the visual cue; and a temporal element attribute that defines 
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temporal characteristics of the visual cue (figure 3, Labels 36, & 44, & Para. 0061, & 
0070, & 01 13). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Bugaj by specifically providing "a visual element 
attribute that defines a visual representation of the visual cue; a spatial element attribute 
that defines spatial characteristics of the visual cue; and a temporal element attribute 
that defines temporal characteristics of the visual cue", as taught by Hendricks, for the 
purpose of providing interactive presentations thereby providing additional information 
on presentation to user while user is watching the presentation. 
5. Claims 34-45 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Bugaj, in view of Hendricks, in view of US Patent Publication 2005/0198569 to Fong et 
al. (Fong). 

Regarding claim 34, Bugaj teaches an XML data structure holding the elements 
of the of the newscast scenario written in XML code comprising elements stored in tags 
wherein each element has its own properly defined attribute including spatial and 
temporal characteristics to perform a presentation (page 27) which meets "retrieving the 
XML-based data from a computer-readable storage medium by using a computer, 
wherein the XML-based data includes an XML-based element for a visual cue together 
with an XML-based element for a visual component contained in the multimedia 
presentation, wherein the XML-based visual cue element is nested within the XML- 
based element for the associated visual component". 
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Bugaj teaches a layout section of a SMIL document including alternative layout 
elements embedded in a switch element used to determine the placement of the 
presentation (section 5, General Semantics, lines 1-6), however, Bugaj is silent on 
wherein the XML-based visual cue element includes attributes that define temporal and 
spatial relativity between a display of the visual cue and a display of the visual 
component; editing the XML-based data; and storing the edited XML-based data in the 
computer-readable storage medium. 

The examiner maintains that it was well known in the art to provide "wherein the 
XML-based visual cue element includes attributes that define temporal and spatial 
relativity between a display of the visual cue and a display of the visual component; 
editing the XML-based data; and storing the edited XML-based data in the computer- 
readable storage medium", as taught by Hendricks and Fong. 

In a similar field of endeavor Hendricks teaches wherein the XML-based visual 
cue element includes attributes that define temporal and spatial relativity between a 
display of the visual cue and a display of the visual component (figure 2, labels 36, & 
40, & Para. 0061, & 0113). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Bugaj by specifically providing "wherein the 
XML-based visual cue element includes attributes that define temporal and spatial 
relativity between a display of the visual cue and a display of the visual component", as 
taught by Hendricks, for the purpose of providing interactive presentations thereby 
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providing additional information on presentation to user while user is watching the 
presentation. 

Bugaj and Hendricks are silent on editing the XML-based data; and storing the 
edited XML-based data in the computer-readable storage medium. 

However, the examiner maintains that it was well known in the art to provide 
"editing the XML-based data; and storing the edited XML-based data in the computer- 
readable storage medium", as taught by Fong. 

In a similar field of endeavor Fong teaches editing the XML-based data; and 
storing the edited XML-based data in the computer-readable storage medium (claim 6, 
entire). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify the method of Bugaj and Hendricks by 
specifically providing "editing the XML-based data; and storing the edited XML-based 
data in the computer-readable storage medium", as taught by Fong, for the purpose of 
debugging thereby allowing for smooth display of encoded presentation. 

Regarding claim 35, Bugaj and Fong are silent on a visual element attribute that 
defines a visual representation of the visual cue; a spatial element attribute that defines 
spatial characteristics of the visual cue; and a temporal element attribute that defines 
temporal characteristics of the visual cue. 

The examiner maintains that it was well known in the art to provide "a visual 
element attribute that defines a visual representation of the visual cue; a spatial element 
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attribute that defines spatial characteristics of the visual cue; and a temporal element 
attribute that defines temporal characteristics of the visual cue", as taught by Hendricks. 

In a similar field of endeavor Hendricks teaches a visual element attribute that 
defines a visual representation of the visual cue; a spatial element attribute that defines 
spatial characteristics of the visual cue; and a temporal element attribute that defines 
temporal characteristics of the visual cue (figure 3, Labels 36, & 44, & Para. 0061, & 
0070, & 01 13). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify the method of Bugaj and Fong by specifically 
providing "a visual element attribute that defines a visual representation of the visual 
cue; a spatial element attribute that defines spatial characteristics of the visual cue; and 
a temporal element attribute that defines temporal characteristics of the visual cue", as 
taught by Hendricks, for the purpose of providing interactive presentations thereby 
providing additional information on presentation to user while user is watching the 
presentation. 

Regarding Claim 36, Bugaj teaches schedule elements with begin and end time 
wherein the duration is the difference between the end time and begin time of an 
element (section 6.1, General Semantics, line 3, & section 6.2 page 8, line 3) which 
meets "a method, wherein the temporal characteristics include at least two of begin 
time, end time, and duration". 



Application/Control Number: 10/637,625 Page 25 

Art Unit: 2423 

Regarding Claim 37, Bugaj teaches an image element tagged as an XML media 
object element (section 6.4, Syntax) which inherently meets "a method, wherein the 
visual representation includes at least one of a shape and a color of the visual cue". 

Regarding claim 38, Bugaj teaches an XML data structure holding the elements 
of the of the newscast scenario written in XML code comprising elements stored in tags 
wherein each element has its own properly defined attribute including spatial and 
temporal characteristics to perform a presentation (page 27) which meets "retrieving the 
XML-based data from a computer-readable storage medium by using a computer, 
wherein the XML-based data includes an XML-based element for a visual cue together 
with an XML-based element for a visual component contained in the multimedia 
presentation, wherein the XML-based visual cue element is nested within the XML- 
based element for the associated visual component". 

Bugaj teaches a layout section of a SMIL document including alternative layout 
elements embedded in a switch element used to determine the placement of the 
presentation (section 5, General Semantics, lines 1-6), however, Bugaj is silent on 
wherein the XML-based visual cue element includes attributes that define temporal and 
spatial relativity between a display of the visual cue and a display of the visual 
component; editing the XML-based data; and storing the edited XML-based data in the 
computer-readable storage medium. 

The examiner maintains that it was well known in the art to provide "wherein the 
XML-based visual cue element includes attributes that define temporal and spatial 
relativity between a display of the visual cue and a display of the visual component; 
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editing the XML-based data; and storing the edited XML-based data in the computer- 
readable storage medium", as taught by Hendricks and Fong. 

In a similar field of endeavor Hendricks teaches wherein the XML-based visual 
cue element includes attributes that define temporal and spatial relativity between a 
display of the visual cue and a display of the visual component (figure 2, labels 36, & 
40, & Para. 0061, & 0113). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Bugaj by specifically providing "wherein the 
XML-based visual cue element includes attributes that define temporal and spatial 
relativity between a display of the visual cue and a display of the visual component", as 
taught by Hendricks, for the purpose of providing interactive presentations thereby 
providing additional information on presentation to user while user is watching the 
presentation. 

Bugaj and Hendricks are silent on editing the XML-based data; and storing the 
edited XML-based data in the computer-readable storage medium. 

However, the examiner maintains that it was well known in the art to provide 
"editing the XML-based data; and storing the edited XML-based data in the computer- 
readable storage medium", as taught by Fong. 

In a similar field of endeavor Fong teaches editing the XML-based data; and 
storing the edited XML-based data in the computer-readable storage medium (claim 6, 
entire). 
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Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify the method of Bugaj and Hendricks by 
specifically providing "editing the XML-based data; and storing the edited XML-based 
data in the computer-readable storage medium", as taught by Fong, for the purpose of 
debugging thereby allowing for smooth display of encoded presentation. 

Regarding claim 39, Bugaj and Fong are silent on a visual element attribute that 
defines a visual representation of the visual cue; a spatial element attribute that defines 
spatial characteristics of the visual cue; and a temporal element attribute that defines 
temporal characteristics of the visual cue. 

The examiner maintains that it was well known in the art to provide "a visual 
element attribute that defines a visual representation of the visual cue; a spatial element 
attribute that defines spatial characteristics of the visual cue; and a temporal element 
attribute that defines temporal characteristics of the visual cue", as taught by Hendricks. 

In a similar field of endeavor Hendricks teaches a visual element attribute that 
defines a visual representation of the visual cue; a spatial element attribute that defines 
spatial characteristics of the visual cue; and a temporal element attribute that defines 
temporal characteristics of the visual cue (figure 3, Labels 36, & 44, & Para. 0061, & 
0070, & 01 13). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify the method of Bugaj and Fong by specifically 
providing "a visual element attribute that defines a visual representation of the visual 
cue; a spatial element attribute that defines spatial characteristics of the visual cue; and 
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a temporal element attribute that defines temporal characteristics of the visual cue", as 
taught by Hendricks, for the purpose of providing interactive presentations thereby 
providing additional information on presentation to user while user is watching the 
presentation. 

Regarding Claim 40, Bugaj teaches schedule elements with begin and end time 
wherein the duration is the difference between the end time and begin time of an 
element (section 6.1, General Semantics, line 3, & section 6.2 page 8, line 3) which 
meets "a method, wherein the temporal characteristics include at least two of begin 
time, end time, and duration". 

Regarding Claim 41, Bugaj teaches an image element tagged as an XML media 
object element (section 6.4, Syntax) which inherently meets "a method, wherein the 
visual representation includes at least one of a shape and a color of the visual cue". 

Regarding claim 42, Bugaj teaches a news broadcast on the web as shown in 
figure 7.1 to the left and right (section 7.4, page 26, lines 2-9) which meets "a computer- 
readable storage medium for storing XML-based data that encodes a synchronized 
display of a multimedia presentation; and for storing computer-executable process 
steps; and a processor for executing the process steps stored in the storage medium". 

In addition Bugaj teaches an XML data structure holding the elements of the of 
the newscast scenario written in XML code comprising elements stored in tags wherein 
each element has its own properly defined attribute including spatial and temporal 
characteristics to perform a presentation (page 27) which meets "retrieving the XML- 
based data from the computer-readable storage medium, wherein the XML-based data 
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includes an XML-based element for a visual cue together with an XML-based element 
for a visual component contained in the multimedia presentation, wherein the XML- 
based visual cue clement is nested within the XML-based clement for the associated 
visual component". 

Bugaj teaches a layout section of a SMIL document including alternative layout 
elements embedded in a switch element used to determine the placement of the 
presentation (section 5, General Semantics, lines 1-6), however, Bugaj is silent on 
wherein the XML-based visual cue element includes attributes that define temporal and 
spatial relativity between a display of the visual cue and a display of the visual 
component; storing computer-executable steps that edit the XML-based data; wherein 
the process steps comprise: editing the XML-based data; and storing the edited XML- 
based data in the storage medium. 

The examiner maintains that it was well known in the art to provide "wherein the 
XML-based visual cue element includes attributes that define temporal and spatial 
relativity between a display of the visual cue and a display of the visual component; 
storing computer-executable steps that edit the XML-based data; wherein the process 
steps comprise: editing the XML-based data; and storing the edited XML-based data in 
the storage medium", as taught by Hendricks and Fong. 

In a similar field of endeavor Hendricks teaches wherein the XML-based visual 
cue element includes attributes that define temporal and spatial relativity between a 
display of the visual cue and a display of the visual component (figure 2, labels 36, & 
40, & Para. 0061, & 0113). 
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Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Bugaj by specifically providing "wherein the 
XML-based visual cue element includes attributes that define temporal and spatial 
relativity between a display of the visual cue and a display of the visual component", as 
taught by Hendricks, for the purpose of providing interactive presentations thereby 
providing additional information on presentation to user while user is watching the 
presentation. 

Bugaj and Hendricks are silent on storing computer-executable steps that edit 
the XML-based data; wherein the process steps comprise: editing the XML-based data; 
and storing the edited XML-based data in the storage medium. 

However, the examiner maintains that it was well known in the art to provide 
"storing computer-executable steps that edit the XML-based data; wherein the process 
steps comprise: editing the XML-based data; and storing the edited XML-based data in 
the storage medium", as taught by Fong. 

In a similar field of endeavor Fong teaches storing computer-executable steps 
that edit the XML-based data; wherein the process steps comprise: editing the XML- 
based data; and storing the edited XML-based data in the storage medium (Para. 0170, 
entire, & claim 6, entire). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify the system of Bugaj and Hendricks by 
specifically providing "storing computer-executable steps that edit the XML-based data; 
wherein the process steps comprise: editing the XML-based data; and storing the edited 
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XML-based data in the storage medium", as taught by Fong, for the purpose of 
debugging thereby allowing for smooth display of encoded presentation. 

Regarding claim 43, Bugaj and Fong are silent on a visual element attribute that 
defines a visual representation of the visual cue; a spatial element attribute that defines 
spatial characteristics of the visual cue; and a temporal element attribute that defines 
temporal characteristics of the visual cue. 

The examiner maintains that it was well known in the art to provide "a visual 
element attribute that defines a visual representation of the visual cue; a spatial element 
attribute that defines spatial characteristics of the visual cue; and a temporal element 
attribute that defines temporal characteristics of the visual cue", as taught by Hendricks. 

In a similar field of endeavor Hendricks teaches a visual element attribute that 
defines a visual representation of the visual cue; a spatial element attribute that defines 
spatial characteristics of the visual cue; and a temporal element attribute that defines 
temporal characteristics of the visual cue (figure 3, Labels 36, & 44, & Para. 0061, & 
0070, & 01 13). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify the ystem of Bugaj and Fong by specifically 
providing "a visual element attribute that defines a visual representation of the visual 
cue; a spatial element attribute that defines spatial characteristics of the visual cue; and 
a temporal element attribute that defines temporal characteristics of the visual cue", as 
taught by Hendricks, for the purpose of providing interactive presentations thereby 
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providing additional information on presentation to user while user is watching the 
presentation. 

Regarding Claim 44, Bugaj teaches schedule elements with begin and end time 
wherein the duration is the difference between the end time and begin time of an 
element (section 6.1, General Semantics, line 3, & section 6.2 page 8, line 3) which 
meets "an apparatus, wherein the temporal characteristics include at least two of begin 
time, end time, and duration". 

Regarding Claim 45, Bugaj teaches an image element tagged as an XML media 
object element (section 6.4, Syntax) which inherently meets "an apparatus, wherein the 
visual representation includes at least one of a shape and a color of the visual cue". 
Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to ANTHONY BANTAMOI whose telephone number is 
(571 )270-3581 . The examiner can normally be reached on Monday - Friday 8-5:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Andrew Koenig can be reached on (571) 272 7296. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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